Enable SPEC SAT for Java sources #18779

DoNotPanicUA · 2022-11-01T15:03:25Z

What

Add missing files and configurations to run SAT SPEC as part of the integration tests.
ℹ️ Tech spec

⚠️ Message for reviewers :

The snowflake source passes all SATs but fails on invalid oath creds. It's not related to this PR, but I'm also working on fixing it.
The scaffold-java-jdbc is not implemented at all. There is no way to add these tests thoroughly.

Database source	Enabled SAT SPEC	Succesfully pass tests	Fix spec format
bigquery	✔️	✔️	➖
clickhouse	✔️	✔️	➖
cockroachdb	✔️	✔️	➖
db2	✔️	✔️	➕
mssql	✔️	✔️	➕
mysql	✔️	✔️	➕
oracle	✔️	✔️	➕
postgres	✔️	✔️	➕
redshift	✔️	✔️	➖
scaffold-java-jdbc	❌	❌	❌
snowflake	✔️	✔️	➕
tidb	✔️	✔️	➖
elasticsearch	✔️	✔️	➕
kafka	✔️	✔️	➕
mongodb-v2	✔️	✔️	➕
sftp	✔️	✔️	➖

⚠️ scaffold-java-jdbc source is not implemented!

github-actions · 2022-11-01T15:04:25Z

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to run corresponding integration tests:

source-mssql-strict-encrypt%0A- source-scaffold-java-jdbc%0A- source-mysql-strict-encrypt%0A- source-cockroachdb-strict-encrypt%0A- source-tidb%0A- source-bigquery%0A- source-redshift%0A- source-db2-strict-encrypt%0A- source-oracle-strict-encrypt%0A- source-cockroachdb%0A- source-snowflake%0A- source-mysql%0A- source-mssql%0A- source-alloydb-strict-encrypt%0A- source-mongodb-v2%0A- source-alloydb%0A- source-db2%0A- source-clickhouse-strict-encrypt%0A- source-clickhouse%0A- source-postgres%0A- source-mongodb-strict-encrypt%0A- source-oracle%0A- source-postgres-strict-encrypt%0A- source-elasticsearch%0A- source-sftp

grubberr · 2022-11-01T16:37:49Z

@DoNotPanicUA @sherifnada

Source acceptance test AND Integration tests mess in airbyte

I have counted 3 ways to run SAT tests:

acceptance-test-docker.sh
python -m pytest integration_tests -p integration_tests.acceptance - from docs
./gradlew airbyte-integrations:connectors:source-github::sourceAcceptanceTest - here - CI way

Ways (2),(3) - are running SAT through Integration tests, but way (1) - runs SAT directly.
That's way (1) can work for you but (2), (3) can fail.

grubberr · 2022-11-01T16:52:40Z

@DoNotPanicUA @sherifnada

to be more concise

For ways (2), (3) we need this file integration_tests/acceptance.py
but way (1) don't use this file integration_tests/acceptance.py

github-actions · 2022-11-01T17:02:16Z

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

Run integration tests
Bump connector version
Add changelog
Publish the new version

⚠ Sources (26)

Connector	Version	Changelog	Publish
`source-alloydb`	`1.0.17`	✅	✅
`source-alloydb-strict-encrypt`	`1.0.17`	✅	⚠ (not in seed)
`source-bigquery`	`0.2.3`	✅	✅
`source-clickhouse`	`0.1.14`	✅	✅
`source-clickhouse-strict-encrypt`	`0.1.14`	✅	⚠ (not in seed)
`source-cockroachdb`	`0.1.18`	✅	✅
`source-cockroachdb-strict-encrypt`	`0.1.18`	✅	⚠ (not in seed)
`source-db2`	`0.1.16`	✅	✅
`source-db2-strict-encrypt`	`0.1.16`	✅	⚠ (not in seed)
`source-elasticsearch`	`0.1.0`	✅	✅
`source-kafka`	`0.2.2`	✅	✅
`source-mongodb-strict-encrypt`	`0.1.19`	⚠ (doc not found)	⚠ (not in seed)
`source-mongodb-v2`	`0.1.19`	✅	✅
`source-mssql`	`0.4.25`	✅	✅
`source-mssql-strict-encrypt`	`0.4.25`	✅	⚠ (not in seed)
`source-mysql`	`1.0.11`	✅	✅
`source-mysql-strict-encrypt`	`1.0.11`	✅	⚠ (not in seed)
`source-oracle`	`0.3.21`	✅	✅
`source-oracle-strict-encrypt`	`0.3.21`	✅	⚠ (not in seed)
`source-postgres`	`1.0.22`	✅	✅
`source-postgres-strict-encrypt`	`1.0.22`	✅	⚠ (not in seed)
`source-redshift`	`0.3.15`	✅	✅
`source-scaffold-java-jdbc`	`0.1.0`	⚠ (doc not found)	⚠ (not in seed)
`source-sftp`	`0.1.2`	✅	✅
`source-snowflake`	`0.1.24`	✅	✅
`source-tidb`	`0.2.1`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector	Version	Changelog	Publish

See "Actionable Items" below for how to resolve warnings and errors.

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the seed file (e.g. `source_definitions.yaml`), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
	❌ diff seed version	The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the `/publish` command to publish the latest version.

evantahler · 2022-11-01T20:52:38Z

airbyte-integrations/connectors/source-bigquery/integration_tests/acceptance.py

@@ -0,0 +1,16 @@
+#


I don't think these /integration_tests files are needed

https://github.com/airbytehq/airbyte/blob/master/buildSrc/src/main/groovy/airbyte-source-acceptance-test.gradle#L22

?

@evantahler
if want to run the tests only locally as we see here, yes.
If you want to run it as part of the CI testing, you have to add it.

The acceptance.py is indeed required to launch the test suite with the SAT plugin. We should find a better way because it's far from ideal. @DoNotPanicUA could you please create an issue about this?

DoNotPanicUA · 2022-11-07T19:24:17Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3413384232
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3413384232
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 12 passed, 5 skipped, 19 warnings in 29.89s ==================

DoNotPanicUA · 2022-11-07T19:24:28Z

/test connector=connectors/source-mongodb-v2

🕑 connectors/source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/3413386008
✅ connectors/source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/3413386008
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 12 passed, 5 skipped, 19 warnings in 27.31s ==================

DoNotPanicUA · 2022-11-07T19:24:38Z

/test connector=connectors/source-jdbc

DoNotPanicUA · 2022-11-07T19:24:48Z

/test connector=connectors/source-elasticsearch

DoNotPanicUA · 2022-11-07T19:25:02Z

/test connector=connectors/source-cockroachdb

🕑 connectors/source-cockroachdb https://github.com/airbytehq/airbyte/actions/runs/3413391835
✅ connectors/source-cockroachdb https://github.com/airbytehq/airbyte/actions/runs/3413391835
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 12 passed, 5 skipped, 19 warnings in 22.05s ==================

DoNotPanicUA · 2022-11-07T19:25:13Z

/test connector=connectors/source-clickhouse

🕑 connectors/source-clickhouse https://github.com/airbytehq/airbyte/actions/runs/3413392058
✅ connectors/source-clickhouse https://github.com/airbytehq/airbyte/actions/runs/3413392058
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 12 passed, 5 skipped, 19 warnings in 31.47s ==================

DoNotPanicUA · 2022-11-07T19:25:24Z

/test connector=connectors/source-bigquery

🕑 connectors/source-bigquery https://github.com/airbytehq/airbyte/actions/runs/3413393883
✅ connectors/source-bigquery https://github.com/airbytehq/airbyte/actions/runs/3413393883
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 12 passed, 5 skipped, 19 warnings in 19.38s ==================

DoNotPanicUA · 2022-11-08T16:39:12Z

Update source-jdbc-source template #19130

…st' into aleonets/17580-enable-py-spec-test

DoNotPanicUA · 2022-11-09T15:59:41Z

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/3429738131
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/3429738131
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:71: Backward compatibility tests are disabled for version 0.4.25.
================= 11 passed, 6 skipped, 19 warnings in 13.60s ==================

DoNotPanicUA · 2022-11-09T15:59:59Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3429740438
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3429740438
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       133      3    98%   87, 93, 230
	 source_acceptance_test/conftest.py                     196     92    53%   35, 41-43, 48, 54, 60, 66, 72-74, 93, 98-100, 106-108, 114-115, 120-121, 126, 132, 141-150, 156-161, 176, 200, 231, 237, 243-248, 256-261, 269-282, 287-293, 300-311, 318-334
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              345    110    68%   53, 64-72, 77-84, 88-89, 93-94, 178, 216-233, 242-250, 254-259, 265, 298-303, 341-348, 391-393, 396, 461-469, 481-484, 489, 545-546, 552, 555, 591-601, 614-639
	 source_acceptance_test/tests/test_incremental.py       145     20    86%   21-23, 29-31, 36-43, 48-61, 224
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/config_migration.py        23     23     0%   5-37
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-68, 71-73, 76-78, 81-83, 86-88, 91-93, 96-114, 148-150
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1512    375    75%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
================= 12 passed, 5 skipped, 19 warnings in 22.86s ==================

DoNotPanicUA · 2022-11-09T19:46:40Z

@alafanechere
#19133

edgao · 2022-12-01T00:47:48Z

airbyte-integrations/connectors/source-postgres/src/main/resources/spec.json

                "order": 0
              },
              "plugin": {
                "type": "string",
                "title": "Plugin",
                "description": "A logical decoding plugin installed on the PostgreSQL server. The `pgoutput` plugin is used by default. If the replication table contains a lot of big jsonb values it is recommended to use `wal2json` plugin. Read more about <a href=\"https://docs.airbyte.com/integrations/sources/postgres#step-2-select-a-replication-plugin\">selecting replication plugins</a>.",
                "enum": ["pgoutput", "wal2json"],
-                "default": "pgoutput",
+                "const": "pgoutput",


was this change intended? It seems to have removed the ability for users to switch to a different plugin https://github.com/airbytehq/oncall/issues/1133

submitted #19985, if this passes SATs then I assume things are fine 🤷

The SAT forced me to use const only.

* enable py spec compatibility tests * add missing properties * use expected spec file instead of the source spec.json + use dummy config file * add missing files for the clickhouse * move test files to the integration-test from the unit test folder * add missing files to cockroachdb * add missing files to db2 + fix spec format * add missing files to elasticsearch + fix spec format * add missing files to jdbc * add missing files to mongodb_v2 + fix spec format * add missing files to mssql + fix spec format * add missing files to mysql + fix spec format * add missing files to postgres + fix spec format * add missing files to oracle + fix spec format * add missing files to redshift * add missing files to sftp * add missing files to snowflake + fix spec format * add missing files to tidb * add missing files to kafka - fix spec format * airbyte-source-acceptance-test added Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * add missing import * Delete acceptance-test-config.yml * Delete acceptance-test-docker.sh * Update build.gradle * Update build.gradle * format * revert changes * manual .sh files format * upd expected spec * format * fix SAT after master merge Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>

enable py spec compatibility tests

838618e

github-actions bot added the area/connectors Connector related issues label Nov 1, 2022

This comment was marked as outdated.

Sign in to view

DoNotPanicUA temporarily deployed to more-secrets November 1, 2022 15:04 Inactive

DoNotPanicUA linked an issue Nov 1, 2022 that may be closed by this pull request

[Spike] Adding SAT testing to Java connectors #17580

Closed

2 tasks

DoNotPanicUA self-assigned this Nov 1, 2022

add missing properties

ce22526

DoNotPanicUA temporarily deployed to more-secrets November 1, 2022 17:03 Inactive

evantahler reviewed Nov 1, 2022

View reviewed changes

This comment was marked as outdated.

Sign in to view

DoNotPanicUA mentioned this pull request Nov 8, 2022

Extend source-java-jdbc by files required for SAT SPEC (Example #18779) #19130

Closed

DoNotPanicUA linked an issue Nov 8, 2022 that may be closed by this pull request

Add SAT to all Java connectors and test just the SPEC command #18556

Closed

DoNotPanicUA added 2 commits November 8, 2022 21:18

upd expected spec

116bf78

Merge remote-tracking branch 'origin/aleonets/17580-enable-py-spec-te…

c67136b

…st' into aleonets/17580-enable-py-spec-test

DoNotPanicUA temporarily deployed to more-secrets November 9, 2022 14:09 Inactive

format

b1ed0da

DoNotPanicUA requested a review from a team as a code owner November 9, 2022 15:41

github-actions bot added the CDK Connector Development Kit label Nov 9, 2022

DoNotPanicUA temporarily deployed to more-secrets November 9, 2022 15:43 Inactive

fix SAT after master merge

235269b

DoNotPanicUA temporarily deployed to more-secrets November 9, 2022 16:01 Inactive

Merge branch 'master' into aleonets/17580-enable-py-spec-test

c5efca9

DoNotPanicUA merged commit 58e4ef6 into master Nov 9, 2022

DoNotPanicUA deleted the aleonets/17580-enable-py-spec-test branch November 9, 2022 19:45

github-actions bot removed the CDK Connector Development Kit label Nov 9, 2022

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

edgao reviewed Dec 1, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable SPEC SAT for Java sources #18779

Enable SPEC SAT for Java sources #18779

DoNotPanicUA commented Nov 1, 2022 •

edited

Loading

github-actions bot commented Nov 1, 2022

This comment was marked as outdated.

This comment was marked as outdated.

grubberr commented Nov 1, 2022

grubberr commented Nov 1, 2022

github-actions bot commented Nov 1, 2022 •

edited

Loading

⚠ Sources (26)

✅ Destinations (0)

Actionable Items

evantahler Nov 1, 2022

grubberr Nov 1, 2022

DoNotPanicUA Nov 2, 2022

alafanechere Nov 4, 2022

This comment was marked as outdated.

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022

DoNotPanicUA commented Nov 7, 2022

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 8, 2022

DoNotPanicUA commented Nov 9, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 9, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 9, 2022

edgao Dec 1, 2022

edgao Dec 1, 2022

DoNotPanicUA Dec 1, 2022

Enable SPEC SAT for Java sources #18779

Enable SPEC SAT for Java sources #18779

Conversation

DoNotPanicUA commented Nov 1, 2022 • edited Loading

What

github-actions bot commented Nov 1, 2022

This comment was marked as outdated.

This comment was marked as outdated.

grubberr commented Nov 1, 2022

Source acceptance test AND Integration tests mess in airbyte

grubberr commented Nov 1, 2022

github-actions bot commented Nov 1, 2022 • edited Loading

Affected Connector Report

⚠ Sources (26)

✅ Destinations (0)

Actionable Items

evantahler Nov 1, 2022

Choose a reason for hiding this comment

grubberr Nov 1, 2022

Choose a reason for hiding this comment

DoNotPanicUA Nov 2, 2022

Choose a reason for hiding this comment

alafanechere Nov 4, 2022

Choose a reason for hiding this comment

This comment was marked as outdated.

DoNotPanicUA commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 7, 2022

DoNotPanicUA commented Nov 7, 2022

DoNotPanicUA commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 7, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 8, 2022

DoNotPanicUA commented Nov 9, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 9, 2022 • edited by github-actions bot Loading

Build Passed

DoNotPanicUA commented Nov 9, 2022

edgao Dec 1, 2022

Choose a reason for hiding this comment

edgao Dec 1, 2022

Choose a reason for hiding this comment

DoNotPanicUA Dec 1, 2022

Choose a reason for hiding this comment

DoNotPanicUA commented Nov 1, 2022 •

edited

Loading

github-actions bot commented Nov 1, 2022 •

edited

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 7, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 9, 2022 •

edited by github-actions bot

Loading

DoNotPanicUA commented Nov 9, 2022 •

edited by github-actions bot

Loading